Skip to content

fix(agent) SWE coding-agent RL stability bugs (abort handling, session cleanup) - #2124

Merged
zhuzilin merged 1 commit into
THUDM:mainfrom
jingshenghang:swe-rl-stability-fixes
Jun 23, 2026
Merged

zhuzilin merged 1 commit into
THUDM:mainfrom
jingshenghang:swe-rl-stability-fixes

Conversation

@jingshenghang

@jingshenghang jingshenghang commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator
  • generate.py: an aborted sample now sets rollout_log_probs=[0.0] and remove_sample=True so it contributes no gradient and is excluded from the GRPO/GSPO group baseline; finally now calls drop_session instead of finish_session.
  • adapters/common.py: add drop_session() for cleanup-only teardown; make finish_session's base_sample a required arg.
  • harness/common.py: run_command tolerates an empty done-marker (truncate-then -write window) instead of crashing on int('').
  • trajectory.py: add TrajectoryManager.drop_session() to drop trees/turn-count idempotently.
  • run_qwen36_35b_a3b_swe_8nodes.sh: switch algo args to grpo eps-clip 0.2/0.28.

Port of gitlab MR !514 (subset present in the MR diff):

- generate.py: an aborted sample now sets rollout_log_probs=[0.0] and
  remove_sample=True so it contributes no gradient and is excluded from the
  GRPO/GSPO group baseline; finally now calls drop_session instead of
  finish_session.
- adapters/common.py: add drop_session() for cleanup-only teardown; make
  finish_session's base_sample a required arg.
- harness/common.py: run_command tolerates an empty done-marker (truncate-then
  -write window) instead of crashing on int('').
- trajectory.py: add TrajectoryManager.drop_session() to drop trees/turn-count
  idempotently.
- run_qwen36_35b_a3b_swe_8nodes.sh: switch algo args to grpo eps-clip 0.2/0.28.
@jingshenghang
jingshenghang force-pushed the swe-rl-stability-fixes branch from 3f3dd31 to ffe4e9f Compare June 23, 2026 06:39
@jingshenghang jingshenghang changed the title Fix SWE coding-agent RL stability bugs (abort handling, session cleanup) fix(agent) SWE coding-agent RL stability bugs (abort handling, session cleanup) Jun 23, 2026
@zhuzilin
zhuzilin merged commit 34a533b into THUDM:main Jun 23, 2026
76 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants